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ABSTRACT 



This paper discusses the traditional lecture-based 
instruction method. Strong experimental evidence suggests that social context 
affects knowledge construction and depends on prior knowledge. Engagement in 
the inquiry process using constructivist viewpoints can make the 
characteristics of science, scientific knowledge, and scientific method easy 
to understand. This study involves first semester college students (n=90) and 
compares students whose science programs are project -based and 
inquiry-oriented to students in traditional science programs. Data collection 
included Likert scale surveys, and results are presented in three categories: 
(1) students' epistemology in science; (2) students' methods of ; justifying 
decisions; and (3) students* attitudes towards and agency in science. 
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Abstract 





A group of 14 first-semester college students in an inquiry-based science program were interviewed pre- and post-semester about: a) their 
experiences in science, b) their beliefs about the nature of scientific knowledge (epistemology), c) their methods of justifying decisions about 
complex science problems, and c) their degree of agency in science. Students who were involved in laboratory, field and library research on 
relevant science problems made greater gains in these developmental domains than did those students described in thie literature in more 

traditional programs. 
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Developmental Measures as Evaluation Tools for Inquiry-Based Science Programs 



Introduction 

The traditional view of lecture-based college instruction as the transfer of knowledge from faculty member to student is not in keeping with 
constructivist conceptions of learning that have emerged from the fields of human development and cognition. There is strong experimental 
evidence that knowledge is constructed by an individual in social contexts, that it is dependent on prior knowledge, and that it is most useful 
when learned in situations similar to those in which it is to be applied (Minick, Stone, & Forman 1993; Lave 1991; Wertsch 1991; Resnick 
1987; Segal, Chipman, & Glaser 1985; Glaser 1984; Vygotsky 1978). College instruction that aims to be aligned with constructivist views of 
learning would expose students to the real methods of the disciplines they are taught, using techniques that can be applied in diverse 
situations in the future. It would allow for interactions with other students, faculty, and support staff as well as with the tools and language 
of the specific discourse community. When necessary, it would also provide background information required for further understanding and 
help students see that they are capable of constructing knowledge in that discipline (Re snick, 1989). 

In terms of science instruction in particular, this constructivist view points to engagement in real science inquiry. In fact, in 1996 the National 
Science Foundation (NSF 1996) proposed decisive action to change science, mathematics, engineering, and technology education that would 
expose American college students to the kinds of experiences that require: 
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All students have access to supportive, excellent undergraduate education in science, mathematics, engineering, and technology, and all students learn these 
subjects by direct experience with the methods and processes of inquiry. 

A report on its Review of Undergraduate Education 

by the Advisory committee to the NSF, 1996, p. 1 

The NSFs unusual foray into the recommendation of specific pedagogy for teaching science evidences the compelling nature of the supporting 
literature on inquiry learning for students’ understanding of the nature of science and science knowledge. 

There are a number of quahties of science which one can better understand by engaging in the process of inquiry. Two important 
characteristics are described here. One is that there is uncertainty in scientific knowledge. Scientific ideas, theories, principles, and laws are 
extrapolations from observations. As such, they are subject to imprecision, error, and doubt even if the observations were done most carefully 
and systematically (Feynman 1998). The uncertainty of scientific knowledge is a limitation of the subject content, but it is not a fatal 
drawback. Richard Feynman, a renown physicist and philosopher of science, says that the doubt inherent in scientific knowledge is of value. 
He states: 



"... if we did not have a doubt or recognize ignorance, we would not get any new ideas. There would be nothing worth checking, because we would know what is 
true. So what we call scientific knowledge today is a body of statements of varying degrees of certainty. Some of them are most unsure; some of them are nearly 
sure; but none is absolutely certain.... Doubt is not to be feared... it is to be welcomed as the possibility of a new potential for human beings." 

- Feynman 1998, pp. 27-28 

A second characteristic of science that becomes clear by engaging in inquiry is that the scientific method is a complex, nonhnear process. In 
doing science, a scientist goes through iterative cycles of observation, generation of hypotheses, testing, evaluation of results, and the making 
of decision-judgments. By attempting to learn science by merely looking at the results of scientific inquiry and argumentation, one does not 
have the same opportunities to weigh evidence and consider alternatives - processes that are at the core of science. In 1916, John Dewey 
wrote of the importance of conflict in stimulating reflective thought. He said: 

"We have reached the point of conflict in the matters of an experience. It is in this conflict and because of it that the matters, or significant [properties], stand 
out os matters. As long as the sun revolves about the earth without question, this "content" is not in any way abstracted. Its distinction from the form or 
mode of experience as its matter is the work of reflection. The same conflict makes other experiences assume discriminated objectification; they, too, cease to be 
ways of living, and become distinct objects of observation and consideration." 

- Dewey 1916, p 136 

By engaging in science rather than learning science facts, individuals have the opportunity to: a) decide what constitutes good evidence and 
hone their abilities to weigh evidence, b) see that they are capable of making science judgments and constructing science knowledge, and c) 
learn that science is a complex endeavor that always involves uncertainty. Teaching science via inquiry, then, fits more closely with the nature 
of science itself and one would expect that students so engaged would learn a great deal about the nature of science and their ability to 
participate in the creation of scientific knowledge. 

Purpose 

Although it is clear that understanding science is not only about learning science content, program evaluation tends to concentrate on student 
performance on standardized tests of content knowledge (e.g. subject area G.R.E’s). Such evaluation ignores some very important aspects 
student learning that are at the heart of inquiry-based learning. Program evaluation in science should be expanded to include a fuller range of 
learning in science. 

The purpose of this study is two-fold. First, it is intended to explore the potential of using three dimensions of adult development 
(epistemology, method of justifying decisions, and agency) to evaluate college science programs. The second purpose of the paper is to 
determine the elements of an inquiry-based science program that most affect students’ understanding of the nature of science, methods of 
justifying their decisions, and feehng ownership over their work. 

Relevant Literature - a brief overview 

In 1970, William Perry and his colleagues published a nine-stage scheme of development of college-aged students’ patterns of thought, 
specifically with regard to the nature of knowledge and authority. A number of researchers and theorists have added to the work of Perry, 
including Belenky, Chnchy, Goldberger, and Tarule (1986), Marcia Baxter Magolda (1992), and Karen Strom Kitchener and Patricia King 
(1994). These theories are complementary. 

Table 1 below shows a synthesis of the work of these theorists as it applies to the college years (Kitchener and King have further 
epistemological stages that apply to more advanced students). Following the table from left to right is the typical pattern of development for 
college-aged students in terms of epistemic beliefs and methods of justifying decisions. The shaded portion of the table represents positions 
in which individuals typically have authority-based thinking; the unshaded portion represents positions of internal agency. 

Notice that until learners are mutualistic thinkers they do not have internal agency. Up until this same point, they do not use evidence to 
justify their decision making about complex problems. As mutuahstic and contextual thinkers their focus shifts to an understanding of the 
processes of inquiry, with appreciation for uncertainty. They can then justify their thinking based on contextually derived evidence and 
understand their roles as active participants in original thinking. 
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Table 1: Adult Development Positions - A Synthesis of Developmental Theories 
Domain Typifies 



Position 


Nature of Knowledge 


Method of Justifying 
Decisions 




Strict 

Dualistic 

Thinking 


Knowledge is certain 


What is observed is true 

No justification 
necessary 




Modified 

Dualistic 

Thinking 


Knowledge is certain, 
but not always 
immediately available 


Decide based on what 
authority teUs you 

Disagreement exists 
because some 
authorities know better 




Transitional 

Thinking 


Knowledge is certain in 
some areas; uncertain in 
others 


When right answers are 
known, decide based on 
what authorities teU you 

When answers unknown, 
decide on what "feels 
right" Go&ic, most recent 
information, etc.) 


Most 

First-Year 

College 

Students 


Mutualistic 

Thinking 


Knowledge is uncertain; 
everyone has their own 
beliefs. (There are many 
possible answers and no 
way of telling which is 
best) 


Decide based on 
idiosyncratic use of 
evidence (often to back 
up own beliefs); 
variables seen as lost 
data, incorrect reporting, 
different access to 
information 


Most 

College 

Seniors 


Contextual 

Thinking 


Known within a 
particular context 

(what is known is 
limited by perspective of 
knower) 


Decide based on evidence 
30 use rules of inquiry for 
that context 

Credibility of sources 
must be weighed in that 
context 


Graduate 
students or 
career 
workers 



Shaded positions are those in which students are authority-based. Unshaded positions are those in which students have internal agency. 

Based on the work of Perry, Belenky et al, Baxter Magolda, and King and Kitchener. Synthesis developed by Laura Wenk, University of 
Massachusetts/Amherst 

According to studies by King and Kitchener (1994) students typically enter college as transitional thinkers and progress to mutualistic 
thinking by the end of their senior year in college. They report movement of a half a stage over the 4 years students spend in college, which is 
not to say that there is generally no movement. Such a change actually represents important shifts in understanding. It does mean, however, 
that many students leave college without the ability to make judgments about complex problems based on the use of evidence. 

The first-semester students at Hampshire College whose science programs is project-based and inquiry-oriented will be compared to the 
students described above in the studies by Perry, Baxter Magolda, and King and Kitchener. 



Hypotheses 



In addition to describing the learning experiences that most affect student learning, this study is designed to specifically test the following 
three hypotheses: 



1 . 

2 . 



o 




that college students whose introductory science experience is inquiry-based will demonstrate more mature epistemological beliefs in 
science than students described in the literature in traditional science programs, 

that college students whose introductory science experience is inquiry-based will demonstrate more mature methods of justifying their 
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decisions in science than students described in the literature in traditional science programs, and 
3. that by the end of their first semester, these students would see themselves as more able to participate in science and would have 
more positive attitudes towards science than at the start of the semester. 



Methods 

Study population 

Students involved in the study were 90 first-semester students at Hampshire College. Their Natural Scien ce program is designed 
to engage students in research on open-ended, problems that can be tackled through litorv. laboratory, and/or field research» 
The emphasis in this program is on learning the process of science. 

Data coHection 

Data collected include (contact author for actual instruments!: 

90 students* responses to a 36-item Likert-scale survey about their attitudes and beliefs about science (matched 
pre and post course). 

^ 90 students* responses to an 18-item Likert-scale survey assessing their own learning in the course (post course 

Q-n!y.k, 

In-depth Interviews with a sampling of 16 students In 4 different inquiry-based courses (pre and post course). 

From these data, it will be possible to describe students* epistemic assumptions in science, their agency with regard to science, 
and their methods of making judgments about ill-structured problems. It will also be possibl e to create a description of the Kinds 
of learning opportunities that seem to have the greatest effects on student learning and thinking. This information Is Invaluable 

in designing programs that give all students the opportunity to learn about the tru e nature of science^ 

Comparisons 

Students* epistemology and methods of justifying decisions are compared to studies reported in the literature as_tVPical ioi 
students in traditional programs. Pre-semester interviews show that students entering Hampshire College are not atypical in 
their epistemological assumptions or in their methods of justifying dedsiona. 

Students* sense of aoencv and their attitudes towards science after one semester of study is compared with their 9q encY_aDd 
attitudes as determined bv surveys and interviews at the beginning of the semester. In addition, their sense of gain | n a n^Jmbor 

of science skills is described from post-semester self-assessment surveys (contact author for samples of surveys). 

Results 

The results are presented separately for descriptions of: a) students* epistemology in science, b) students* methods of justifying decisions, and 
c) students’ attitudes towards and agency in science. 

Epistemology in Science: 

Developmental changes are not fast, and different students will respond differently to the same experiences. Typical developmental schemes 
were created by interviewing large numbers of students over time. Interviews of first-year Hampshire students, by comparison, showed 
similar epistemological positions at the beginning of the semester. At the end of one semester, the Hampshire students interviewed, however, 
were often at the epistemological position described for college seniors or for first-year graduate students. That is, Hampshire students 
typically moved from transitional thinking to mutualism, and some showed evidence of contextual thinking. See table 2. 

Some responses to the pre- post-semester Science Questionnaire indicated that more students were thinking mutualisticaUy, that is, they 
were more apt to believe that scientists back up their beliefs by idiosyncratic use of evidence. More students answered that "scientists know 
what the results of their experiments will be before they start" at the end of the semester than at start and more students responded that 
"scientists back up their own ideas by playing with the statistics," although neither change was significant. 

Interestingly, for students who were in classes that faculty characterized as requiring students to read a great deal of primary literature, 
there was significant change in students’ belief to more strongly agree that scientists play with the statistics. Although this is not an 
intended outcome of having students read primary literature (it is in fact, just the opposite), understanding generalizable developmental 
schemes helps make sense of such a result. It means that more students are thinking mutualisticaUy at the end of one semester of science 
than at the beginning, since mutualistic thinkers often believe that "experts" make decisions by idiosyncratic use of evidence. 

Table 2: Comparison of Hampshire College First-Year Students with Students in Traditional Programs 
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Position 


Traditional 
College Programs 
-All Four Years 


Hampshire 

College 

First-year 

Students 


Strict Dualistic 
Thinking 






Modified Dualistic 
Thinking 






Transitional Thinking 


Most First-Year College 
Students 


Most incoming first-year 
students. 

Some 

end-of-first-semester 

students 


Mutualistic Thinking 


Most College Seniors 


Some 

end-of-first-semester 

students 


Contextual Thinking 


Graduate students or 
career workers 


Evidence in some 

end-of-first-semester 

students 



Typically, in interviews at the end of the semester, students were either mutualistic thinkers or showed evidence of a movement towards 
mutualism. A few students used contextual thinking at times. A typical response to queries about whether there are right answers in science 
is exemplified by the following quotes from two students. The first said: 

”I don’t think that in more and more areas of science the right answer is known. Definitely not. I think that more and more information is out there and I think 
there is more and more to choose from and more and more basis to form your own opinion. I think it’s all about opinions and not really about truths so much. 



And similarly, the second stated: 

"Science is never a fact, it is more of educated hypotheses based on what data you do have and just trial and error - figuring out what works and what doesn't. 
All of science is theory. So, it’s not like there are some things that have been proven in science and can never be changed because it’s fact and then there are 
some things that can be questions. I think scientists are always just working to improve upon previous hypotheses and theories and ideas." 

Students who had courses that used a good deal of primary literature and that had students engaged in field or laboratory research showed 
evidence of contextual thinking. That is, these students had concrete experiences that demonstrated the importance of the context in which an 
experiment is conducted and the importance of (and limitations oO the knower. This kind of thinking is generally described for students in 
graduate study or in the first year on the job. 



One student describing the reasons for disagreement among scientists on the question of the safety of food additives, drew on his own 
experiences in a laboratory course. He said: 



"When we were testing [a nearby town's] water, if we turned on the water right away and we took a water sample and read it, the chemicals in there were 
enough to kill somebody after a couple of years. But then if we let it run for 5 minutes, the chemicals were a little bit higher than they should have been. So, 
right there’s just some example of how things can change." 

Similarly, reading primary literature gave students clear ideas of the importance of context. Here’s one student’s description of the 
experience: 



o 

ERIC 



"[Reading the primary literature] definitely gave you a much better flavor for what was going on. The books would say "for years humans have been unsure 
what their relationships are with chimps and gorilla." And then you would go and read the primary literature on that subject, and they lay out exactly how 
they did it, exactly how they did the experiment that would then determine how closely related we all are, they analyzed their findings, they analyzed past 
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primary literature so you kind of get this very, very, very in-depth [view]." 

These are not views of science that are typical of students in traditional science programs (King and Kitchener, 1994; Baxter Magolda, 1992). 
Methods.o.f_J.us.tifaing Decisions in Science: 

One student made clear the connection between an appreciation of uncertainty in science and a need to be able to make decisions based on 
evidence (as opposed to more blindly believing an outside authority). This student said: 

"...you shouldn’t go into science expecting there to be right answers anywhere and that all you should rely on is what you’ve observed yourself and the 
conclusions you’ve drawn from that. And that if you want to accept these other ‘right answers’ or these other ‘truths’, you just got to be careful about doing that, 
and make sure you know how people went about finding it out." 

Self-assessments of gains in ability as a result of their first-semester science course point to greater use of evidence in making scientific 
decisions. Students reported substantive gains in their abilities to: 

Use scientific evidence to support their ideas, 

Engage in laboratory, field, or library research to answer their questions. 

Locate information that would help them to answer a scientific question. 

Critically evaluate a primary research article. 

Make judgments about science issues they might read about in the newspaper. 

Recognize the difference between scientific observations and interpretations 

Such reporting, while not in itself indicative of the extent to which students actually engage in such activities, at least reveals that students 
recognize the importance of these skills and that they have been asked to engage in them in their courses. 

Interview data give a much more detailed picture of the ways in which students use scientific evidence and the pedagogical practices that 
helped them learn to support their ideas. In describing the most important learning experience for her in her first-semester course, one 
student said: 



"The most valuable skill [from my lab course] was being able to analyze and hypothesize what I thought was happening - gaining the skills necessary to 
evaluate [a complex situation] and be able to make an educated guess and then use data to support that." 

It was clear that students who were using evidence to justify their decisions had learned to do so by reading and evaluating primary 
literature. One student gave the following example of how she would decide which study to believe: 

”by reading the abstract and seeing how the experiment was conducted. Because if they only used 10 subjects then their results are probably a lot less reliable 
than somebody who used 50 subjects... And I'd be much more inclined to go with the study that used 50 people... What makes research valid is how 
thoroughly it’s conducted and how carefully, and if the scientists are taking all the possible variables into consideration and just really being thorough in their 
work." 

And in discussing the important features of the inquiry-based course for her learning, the same student said: 

"I had to provide some pretty solid research and work to be able to convince [my professor] otherwise. All the projects and trips, I had to really support what I 
was saying and completely back it up, even if I was just making a guess at something, [my professor] always expected me to have all this, not proof, but just 
support. I got a much better idea of what it means to actually be a scientist and come up with your ideas." 

These views are unusual in first-year college students are clearly attributable to inquiry-based learning where students are engaged in real 
projects and read primary literature. 

Attitudes towards Science and^^rency in Science: 

Comparisons of pre and post semester responses for 90 first-semester students on an attitude and belief survey showed either no change or a 
general trend of improved attitudes towards science. The only questions where there was significant change (p= or < .05) were ones in change 
was toward improved attitudes toward science, greater sense of agency in science, or greater appreciation of scientific thinking. 

Students agreed more strongly with these statements: 



Even if I forget the facts. I’ll still be able to use the thinking s kills I’ve learned in science. 
The process of writing in science has helped me understand scientific ideas. 

I can back up my ideas in science. 

Students disagreed more strongly with this statement: 
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Scientists publish their work in professional journals that are too technical for me to understand. 
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One notable exception to this positive trend was in the significant difference in students’ responses to one question. After a semester of 
science, students answered more strongly that they become bored when scientific issues are raised in casual conversations. Yet in the same 
questionnaire, more students responded that they often watch science programs on television or read articles about science topics in the 
newspaper. 

In self assessments at the end of one semester, students in Natural Sciences courses at Hampshire responded that they made substantial 
gains in their ability to do the following: 

Gained a lot 



Use scientific evidence to support their ideas. 

Engage in laboratory, field, or library research to answer their questions. 
Write about a science topic of interest to them. 

Locate information that would help them to answer a scientific question. 
Critically evaluate a primary research article. 



Gained a fair amount 

Make judgments about science issues they might read about in the newspaper. 

Apply something they learned in a science course to your own Life, 

Make an argument about a controversial point in this field. 

Interpret the tables and graphs they come across in the work of others. 

See connections between science and other fields of study. 

Represent experimental results quantitatively. 

Read a scientific research paper and summarize its main points. 

Recognize the difference between scientific observations and interpretations 

Interview data also suggest that at the end of the semester, students have internal agency in science. All students spoke of their projects as 
feeling like their own work. For example, one student, in discussing this sense of ownership said: 

”In [my Natural Science course] we had a giant final paper that was definitely my own work. I put a ton of time into it, and a lot of time revising it and finding 
references and... didn’t work with anybody else on it. I did all research on it by myself. So I consider that to be my own work." 

Another student, in describing how scientists work (herself included) said: 

"Through experimentation or looking at research that others have done and finding where there might be a little shaky statement or something might not have 
been conducted properly or in the most effective manner. So, basically you’re improving on the work of others and you’re taking your own initiative." 

Students producing papers based on library research, field research, or laboratory research on real- Life problems of their own choosing felt 
ownership over their work and saw themselves as able to make decisions in science. 



Conclusion 



This first-year science program that engages students in real science via lab, field, and library research using primary literature gave 
students a better understanding of the nature of scientific knowledge, the ways to justify decisions based on the rules of inquiry for the field, 
and a better sense of their ability to engage in science than students have as a result of more traditional programs. As one student 
appropriately explained: 



"I think the lecture stuff you learn what the science is and all the other stuff, the field work and the group work and the group research you actually feel what it 
is and you kind of understand it better." 



Comparing the experiences of Hampshire College students to that of students in traditional science programs helps to clarify some of the 
reasons that contextual thinking is a characteristic of graduate students, but not of most college students - even in their senior years. For 
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Hampshire College students, it was by actively engaging in the process of science, that is, developing questions, appropriate means to answer 
those questions, gathering and weighing evidence, as well as engaging in the discourse community of scientists that students understood the 
true nature of science. 

If all students are to understand the nature of science and be able to make decisions about complex issues in science (regardless of whether or 
not they pursue science in advanced study), then perhaps all students should have the opportunity to engage in inquiry-based science courses 
even at the introductory level. In this way, more students might leave college being able to make decisions about the myriad of scientific 
issues in their lives based on the rules of scientific inquiry. 

This is not to suggest that developmental measures be used to assess students. Development is slow and unreliable. Rather, the study 
suggests that involvement in inquiry helps students understand the real nature of science. It also demonstrates that evaluating a program by 
looking at developmental measures is a valuable endeavor that highlights some important benefits of inquiry that might be overlooked when 
one evaluates a program by looking at student performance on content-based tests. 
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